Rethinking Literate Programming in Statistics

Literate programming is becoming increasingly trendy for data analysis because it allows generating dynamic analysis reports for communicating the data analysis and eliminates the chance of untraceable human errors in analysis reports. Traditionally, literate programming includes two separate processes for compiling the code and preparing the documentation. In this article I argue that while this workflow might be satisfactory for software documentation, it is not ideal for writing statistical analysis reports and instead, these processes should run in parallel. I also introduce the weaver package that attempts to examine this idea for Stata software by creating a new log system in HTML or LTEX that can be used simultaneously with Stata log system. The new log system provides many features that the Stata log system lacks such as a built-in syntax highlighter, rendering mathematical notations, inserting figures, creating publication-ready dynamic tables, as well as styling text. The weaver package also produces dynamic PDF documents by converting the HTML log to PDF or typesetting the LTEX log and thus provides a real-time preview of the document without re-compiling the code. The article also discusses the potential applications of weaver package.